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Abstract 

Computing circuits composed of noisy logical gates and their ability to represent arbitrary 
Boolean functions with a given level of error are investigated within a statistical mechanics set- 
ting. Bounds on their performance, derived in the information theory literature for specific gates, 
are straightforwardly retrieved, generalized and identified as the corresponding typical-case phase 
transitions. This framework paves the way for obtaining new results on error-rates, function-depth 
and sensitivity, and their dependence on the gate-type and noise model used. 

PACS numbers: 89.70.-a, 02.50.-r, 89.20.Ff 
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Noise is inherent in most forms of computing and its impact is more dramatic as the 

n 

computing circuits become more complex and of large scale [1]. Classical computing cir- 
cuits based on electromagnetic components suffer from thermal noise and production errors, 
quantum computers suffer from decoherence, whilst an understanding of noisy processes, 
inherent in neural networks and biological systems, remains poorly understood. 

The first model of noisy computation was proposed by von Neumann Q who used Boolean 
circuits composed of e-noisy gates to gain insight into the robustness of biological neuronal 
networks. A circuit in this context is a directed acyclic graph in which nodes of in-degree 
zero are either Boolean constants or references to arguments, nodes of in-degree k > 1 are 
gates computing Boolean functions of k arguments and nodes of out-degree zero represent 
circuit outputs. A formula is a single-output circuit in which the output of each gate is input 
to at most one gate. An e-noisy gate computes a Boolean function a : { — 1, l} fc ^{— 1, 1}, 
but for each input S G { — 1, l} k there is an error probability e such that a(S) — > —a(S)] we 
consider the error probability to be independent for each gate. A noisy circuit with e > 
represents a given deterministic function with a maximum error probability 5 over all possible 
circuit inputs determining its reliability. Von Neumann showed that reliable computation, 
with 5 < 1/2, is possible \l\ for small e values and specific gates, and demonstrated how 
reliability can be improved using e-noisy gates only. In a more recent analysis Pippenger 
demonstrated that formulae only compute reliably up to a certain threshold in the gate 
error rate, and that reliable computation with noisy elements requires strictly greater depth. 
These bounds have subsequently been refined (4, 5, 6], and developed to include circuits J]. 

Random Boolean functions play an important role in information theory as they allow 
for the exploration of average case properties in contrast to the traditionally-studied 
worst-case scenario. The generation of typical functions, sampled uniformly over the space 
of Boolean functions, is a research area in its own right as most conventional methods focus 
on the ability to construct arbitrary functions using basic gates or procedures, but typically 
result in highly uncharacteristic functions when generated at random j^, 10]. Here we 



use a growth process where one defines an initial distribution over a set of simple Boolean 
formulae; these are then combined repeatedly by Boolean connectives to define new formulae. 
One such process 11] uses only a single Boolean connective to show that, under very broad 
conditions, the probability of random functions computed by formulae of depth I tends to 
the uniform distribution over all n- variable Boolean functions as I— >oo 
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FIG. 1: (Color online) The model of two coupled systems with identical topology and different 
inverse temperatures (5 and /3^oo. Gates are indicated by circles and S 1 by squares. 

In this Letter we show how models of random formulae can be mapped onto a physical 
framework and employ methods of statistical physics, developed specifically to analyze the 
typical behavior of random disordered systems, to gain insight into the behavior of noisy 
Boolean random formulae. The stability of the circuit towards input-layer perturbations and 
its dependence on the input magnetization are studied to establish the main characteristics of 
the generated formulae. To investigate the properties of noisy circuits we consider two copies 
of the same topology with different temperatures (1//?), representing the noisy {(3 <oo) and 
noiseless ($— >oo) versions of the same circuit. We show that the typical-case macroscopic 
behavior observed corresponds straightforwardly to the bounds obtained in the information 
theory literature for specific cases [2]-^|. Being very general, the framework is extended 
to consider further properties of random Boolean formulae for different gates and their 
dependence on error level and formula depth. 

The noisy computation model considered here, shown in Fig. [TJ is a feed-forward layered 
Nx(L + l) Boolean circuit. The layers in the circuit are numbered from (input) to L 
(output). Each layer i G {1,..,L} in the circuit is composed of exactly N e-noisy, k-ary 
Boolean gates. Due to gate-noise, the i-th gate in the £-th layer operates in a stochastic 
manner according to the microscopic law 
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where f3 relates to the gate noise e via tanh/3 = 1 — 2e. The gate-output Sf is completely 
random/deterministic when (3 — >0/oo, respectively. The model is acyclic by definition so 
that given the state of the layer i the gates of layer £+1 operate independently of each other. 
This suggests that the probability of the microscopic state 5°, .., S L , where S e E {-1, 1}* 



is a product of ((Tj) over circuit sites and layers. The joint probability of microscopic states 
in two systems of identical topology but different gate-noise is 

L 
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where 



P(S \S )=||- — — ^ -J- — ~i C f_i\i ' (^) 

The adjacency tensor A^ 1 • = 1 when it encodes a connection from outputs ji,..,jk in 
layer £ — 1 to gate-input i in layer £, and otherwise; with {i — 1, .., N;£ — 1,. .,£,}. The 
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conditional probability P(S \S ) is the same as (J3J) but with (3— >/3. The source of disorder 
in our model are the random connections and boundary conditions. Random connections 
are generated by selecting the i-th gate at layer i and sampling exactly k indices, which 
point to outputs of layer £ — 1, uniformly from the set of all possible (unordered) indices 
{ix, ..,ik}- This is carried out repeatedly and independently for all gates and layers giving 
rise to the adjacency tensor probability P(A e ^ ■) = -^5 A t,i ,+(l — -^)8 A e,i n . To cater 
for a possible higher level of correlation, the 0-layer boundary conditions are generated by 
selecting randomly members of the finite set S 1 = {S[, .., S^}; the indices Xj are sampled 
uniformly with P{xj) = 1/n and assigned to the input layer. This leads to the random 
boundary conditions P(S°, S \S J ) = Ylf =1 5 s o. s i 5§ 
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The structure o 
Ising spin systems 



the probability distribution (j2J) is similar to the evolution of disordered 



12] if layers are regarded as discrete time-steps of parallel dynamics. The 



generating functional method [13| provides 

Z[^\ = (e-^M s f+^f}^ , (4) 

where (...) denotes the average generated by (j2J). The generating functional (jlj), re- 
garded also as a characteristic function, is used to compute moments of (j2J) by tak- 
ing partial derivatives with respect to the generating fields {ipf,tpj'}, e.g. (SfSj) = 
— lim^ ^ q Q p - Z[if>; ij)}. To compute Z[ip;ip] we assume that for N — > oo the sys- 
tem becomes self-averaging, i.e. Z = Z, where 7TT is the disorder average. Furthermore, the 
normalization property Z[0; 0] = 1 allows one to average over the disorder Z directly, giving 
rise to the macroscopic observables 
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the average layer activity (magnetization) m(£) on layer £ and overlap C{1) between the 
two systems. Averaging (j3J) over the disorder leads to the saddle-point integral Z[. . .] = 
/ {dPdPdndtl}e N ^ P ' P ' n ^ where * is 

< 5,5 

+ ^P(m) log £ M m [{^,^}] (6) 

m {S e ,S e } 

and M m is an effective single-site measure (after removing the fields ip, ip) 
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(7) 



For N—>-oo the averaged generating functional is dominated by the extremum of Func- 
tional variation with respect to the order parameter P e (S e , S e ) provides the saddle-point 
equation P e (S, S) = ^ m P{fn) \^s e ;S^s e -s) j wnere (' ' ' )-W m is the average with respect 
to (J7J). The physical meaning of P e (S, S) relates to the averaged joint probability of nodes 

linijv-»oo ^ Si=i (fis?;sds?;s) \s*i while the conjugate order 



in the two systems P (S,S) 
parameter, which ensures normalization of P l (S,S), vanishes. This simplifies our effective 
measure (J7J for computing the macroscopic observables, yielding 

A: 
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{5 J ,5 J }i= 1 

x tanh[/3a(5'i, .., S^)] tanh[/3a(Si, .., S^)]. 



{ 1 + ^m(£) + Sjm(l) +S J S j C{£) } 



(8) 



The magnetization m(£) is computed by (jSJ) using /3; initial conditions are m(0) = m(0) = 

]sV Zlses^' C(0) = 1. 

The connectivity profile considered here results in a simple set of equations. The macro- 
scopic behavior of the two systems is completely determined by the set of observables 
{m(£),m(£),C(£)} through the order parameter P\S, S) = Ul + Sm(£) + Sm(£) + SSC(£)), 
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while the single system behavior is dominated by {m(£)}. Furthermore, since (Y[ m S e im ) — ► 
Yimi^i ) f° r fi n he m, the spins in layer £ are uncorrelated when N^oo; this is due to the 
fact that the i-th site is a root of a full fc-ary tree, which grows from the input layer and 
points to Boolean variables in the set S 1 . Loops in the circuit are rare, so that trees can 
be regarded as random independent Boolean formulae for a given input. The output of a 
typical formula at layer £ is determined by P e (S). 

The order parameter C(£) and the normalized Hamming distance D(£) between states 
S e and S are related via the identity D{£) = — C{£)). This gives rise to the measure 
A(£) = linig 5^ D(£), for the circuit's sensitivity with respect to its input. The probability 
P(Sf 7^ S £ i) for any node, which relates to the Hamming distance D(£), facilitates the 
estimate of the noisy circuit's £-layer error probability S(£) =max s / lims^ D(£), comparing 
the noisy and noiseless node values for all inputs. Obviously, in the absence of noise 5{£) = 
0, W. 

To obtain results for a specific case, which could be compared against those obtained 
in the information theory literature, we apply equations (jSJ) for a particular Boolean gate 
a, the fc-input majority gate (MAJ-k). The reasons for choosing this gate are twofold. 



Firet l y , it w as proved B, 9 to be opti-na! for oo iSy conation in formul ae. Second,, a 
formula constructed at random using majority gates can in principle compute any Boolean 
function with uniform probability. A convenient representation of the MAJ-k gate is 
of the form MAJ(Si, .., Sk) = sg n Ej=i Sj) "with odd k. For the particularly simple example 
MAJ-3 one obtains for (3 — > oo 

m(£+l) = ^tanh/3[3m(£)-m 3 (£)] (9) 
C(£+l) = tanh/3 ^m{£)m{£)-^C{£)m 2 {£) 

- \c{£)m\£) + \c{£) + \c\£)] . (10) 

Insight on the functions implemented and the gate noise threshold can be obtained from 
equation @, which describes the evolution of the magnetization from layer to layer. When 
expanded around the stationary solution m(oo) = it identifies the critical noise value 
e* = 1/6, identical to the results of , below which the (unordered) m(oo) = solution 
becomes unstable and two stable (ordered) solutions m(oo) = ^\j lift emerge. Studying the 
joint dynamics of (I91 TTU]) shows that for e> 1/6 the magnetization decays to (exponentially) 
while for e<l/6 the stationary solutions appear, corresponding to the positive and negative 



initial magnetizations m(0), respectively. The boundary separating these phases, shown 
in Fig. identifies the noise-level below which the circuit can preserve one bit of input 
information S 1 = {S} for arbitrarily many layers; the error probability P e (—S) — Sm(t)) 
measures how well it is preserved after i layers. Less complicated functions (fewer layers) 
can be computed with higher gate noise. 

The analysis can easily accommodate other gates, in particular MAJ-fc. Using similar 
arguments one identifies the critical noise level e* = 1 /2-2 fc ~ 2 / ( (fc fc 7} /2 ) below which two stable 
solutions emerge. Computing formulae with limited error 5 above the critical noise level e* 

n 

identical to the threshold reported in [6J, becomes infeasible. Similarly, the noise threshold 
for formulae constructed of NAND gates identifies a threshold noise level e* = (3->/7)/4, 
identical to the one derived in 1141 ]. 

General properties of average formulae can be straightforwardly obtained from the site 
probability of average formulae P e (S) at layer I. Stationary solutions in the noiseless case 
show m(oo) = =t£L, in correspondence to the sign of the initial magnetization; giving rise 
to biased function outputs. For m(0) = one obtains m(oo) = 0, so that each site of the 
model can be associated with some random Boolean function output, evaluating to ±1 with 
equal probability. Consequently, depending on the initial conditions, formulae converge to 
a single Boolean function or to the uniform distribution over some set of functions 
Our result is consistent with majority gate growth process 0, [ll| where for input S 1 = 
{— 1, 1, S[, .., Sn,— S( , S^} stationary state formulae compute all Boolean functions of n 
variables while for S 1 = {—1, 1, S[ , .., S^} (also without —1, 1) they converge to the MAJ-n 
function (odd n) or to the uniform distribution over slice functions (even n) Q. Convergence 
to the stationary solution m(oo) is at depth 0(log n) for m(0) = l/n where n G N in agreement 
with [3]. 

Function error-rates can be calculated through the study of equation ([TO]) describing 
the evolution of the overlap between the two systems. Initial conditions are the same for 
both systems m(0) =m(0) and C(0) = 1. The magnetization in the noisy system (e < 1/6) 
converges to m(oo) = ±y }E§f , depending on the sign of m(0). Using these stationary values 



and equation (TTUD we find C(oo) (7-18 e)-(l-2 e) C 3 (oo) = ±6y/(l-2 e) (1-6 e) leading to 
the error probability 5(oo) plotted in Fig. 

The stationary solution C(oo) = 1 of equation (fTO]) for initial conditions m(0) =0, C(0) = 1 
and e = is unstable under perturbations to C(0), resulting in the stable stationary state 
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FIG. 2: (Color online) Properties of the MAJ-3 gate: (a) Magnetization m and output error 5 as 
a function of gate noise e. (b) Sensitivity of A(£) to input mismatch A(0) for m(0) = 0. (c) Phase 
diagram for gate noise e at layer L. For the MAJ-7 function, evolution of: (d) Magnetization m. 
(e) Output error 5. 

C(oo) = 0. Consequently, the circuit is input-sensitive leading to an increasing Hamming 
distance A(£) for small perturbations A(0) as shown in Fig. [2b- For e > the circuit amplifies 
the noise and S(L) grows but remains limited for sufficiently small e as shown in Fig. Eb. 

To examine the computation performed at layer £ we consider the input set S 1 = 
{S\, .., SV}, corresponding to the function MAJ-7 for the noiseless case, with lowest pos- 
sible initial magnetization m(0) = 1/7 where changes between layers are smallest. Figure [21i 
shows the magnetization m(£) for different gate- noise levels; the convergence-rate decreases 
with increasing e. Close to the critical value the difference equation © can be approximated 
by the differential equation 4fm{£) = -m(£) + |(l — 2e)[3m(£)— m 3 (£)} for continuous £. Its 
solution close to the phase boundary, obtained by expanding e— 1/6 + Ae where |Ae| <C 1, 
exhibits exponential convergence \m(£)—m(oo) \ ~ e -constAei 

The function error 5(£), shown in Fig. [2fe for different e values, exhibits two distinct 
stages in the dynamics. Initially, the error increases until it reaches its maximum value 
at £ = 5, before the MAJ-7 function is computed exactly at £ = 8 for e = (see Fig. [2Jl); 
the location of this maximum is independent of e. This suggests that gate-inputs at layers 
£ < 5 are non-uniform, contributing to noise-amplification, but become more uniform later 
leading to noise-suppression and decreasing error. As we approach e* the number of layers 
needed for the error to reach stationarity increases; in the region e = 1/6 ±Ae it can be 
estimated from the asymptotic form derived for m(£). The dynamic behavior of the error 
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changes to monotonically increasing at e° = \ 



l-m' 2 (0) 
3-ro 2 (0) 



above which noise cannot be reduced 



by additional layers. For e ^> 1/6 the error evolution becomes strictly monotonic it relaxes 
to its stationary value 1/2 exponentially fast. 

By mapping the problem of noisy computation onto a physical framework, we retrieved 
many of the existing bounds and extended them to include arbitrary gates and/or distri- 
bution of gates. In addition, we calculated the level of error and function-bias expected at 
any depth, the sensitivity to input perturbations and expected convergence rate depend- 
ing on the input bias, gate properties and gate-noise level. This framework enables one to 
discover typical properties of noisy computation that are inaccessible via traditional meth- 
ods of information theory and will undoubtedly contribute to exciting new discoveries. For 
instance, one can show that systems composed of the biologically-inspired perceptron-like 
gates are more robust against gate noise than other logical gates and study the effect of hard 
(systematic) noise. 

Support by the Leverhulme trust is acknowledged. 
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